Video terminal device and method of detecting direction of gaze

ABSTRACT

A video terminal device includes a capture module, a detection module, a transmitting module, and a displaying module. The capture module takes images of the eyes of a user when the user is gazing at a display device to obtain three points of focus of the eyes, and confirms a circle bounding the three points of focus. The detection module detects whether an area of the applied circle is less than a first threshold. The transmitting module outputs coordinates of the center of the applied circle as an eyes direction when the area of the circle is less than the first threshold, enabling the participants in a conference to see the eyes direction of the presenter and the presenter to see the eyes directions of each of his audience.

FIELD

The disclosure relates to network communications, and particularly tovideo terminal device and method of detecting eye direction.

BACKGROUND

When a long-distance meeting is needed, people are increasinglyaccustomed to using video conferencing, which can achieve goodcommunication. However, during the video conferencing, participants maynot accurately understand the meaning expressed by speakers or misjudgewhat is being said. For example, when a presentation needs to be shared,although each participant in any place can see content of thepresentation, the speaker can only verbally indicate the location ofsome argument or point in the presentation, even using a laser pen maynot be clear enough, so the participants will not really understand thebriefing or flow of ideas expressed by the speaker.

BRIEF DESCRIPTION OF THE DRAWINGS

Many aspects of the present embodiments can be better understood withreference to the tracking drawings. The components in the drawings arenot necessarily drawn to scale, the emphasis instead being placed uponclearly illustrating the principles of the present embodiments. Likereference numerals designate corresponding parts throughout the severalviews of the drawings.

FIG. 1 is an application environment diagram of one embodiment of avideo terminal device as disclosed.

FIG. 2 is a function module diagram of one embodiment of the videoterminal device of FIG. 1.

FIG. 3 is a schematic illustration of circle in different situations ofcapture module obtaining three visual focus positions.

FIG. 4 is a flowchart of one embodiment of a method of the videoterminal device output eye direction coordinate.

FIG. 5 is a flowchart of one embodiment of a method of the videoterminal device used by listeners handling the eye direction coordinateof a speaker

FIG. 6 is a flowchart of one embodiment of a method of the videoterminal device used by the speaker handling the eye directioncoordinate of the listeners

DETAILED DESCRIPTION

The disclosure is illustrated by way of example and not by way oflimitation in the figures of the accompanying drawings in which likereferences indicate similar elements. It should be noted that referencesto “an” or “one” embodiment in this disclosure are not necessarily tothe same embodiment, and such references mean “at least one.”

The word “module” as used hereinafter, refers to logic embodied inhardware or firmware, or to a collection of software instructions,written in a programming language, such as, for example, JAVA, C, orassembly. One or more software instructions in the modules may beembedded in firmware such as in an EPROM. It will be appreciated thatmodules can comprise connected logic units, such as gates andflip-flops, and may comprise programmable units, such as programmablegate arrays or processors. The modules described herein may beimplemented as either software and/or hardware modules and may be storedin any type of computer-readable medium or other computer storagedevice.

FIG. 1 is an application environment diagram of one embodiment of avideo terminal device 10. In at least one embodiment, the video terminaldevice 10 can be connected to a plurality of peripheral devices thatcomprises a image capture device 20, a display device 30, and a voicein-out device. The video terminal device 10 is connected to an Internet50. The shooting device 20 and the display device 30 can handle imagesfor video conferencing, the voice in-out device can receive or outputaudio. In at least one embodiment, the video terminal device 10 cancapture images of human eyes to determine the directions or points whereparticipants of the video conferencing are looking. In at least oneembodiment, the peripheral devices can also be part of or be integratedwith the video terminal device 10.

FIG. 2 is a function module diagram of one embodiment of the videoterminal device 10 of FIG. 1. In at least one embodiment, the videoterminal device 10 comprises a capture module 210, a detection module220, a transmitting module 230, a displaying module 240, a storagesystem 250, and a processor 260. The modules 210, 220, 230, 240 comprisesoftware programs which are stored in the storage system 250, and theprocessor 260 runs the software programs to achieve functions of thesemodules 210, 220, 230, 240.

In at least one embodiment, during the video conferencing, the functionmodules of the video terminal device 10 takes images of the participantsto determine eye directions of the participants. In particular, an eyesdirection of a speaker regarded as a first perspective is determined bythe video terminal device 10, and an eyes direction coordinates thatcharacterize the eyes direction can be shown in the display device 30.Thus, listeners can see the points or direction of the eyes of a speakerto better understand what the speaker is saying.

In at least one embodiment, when the speaker explains contents of thedisplay device 30 (the contents of the display device 30 also can bebriefing, the following the contents of the display device 30 will bereplace by the briefing), some important issues need to be stated sothat the speaker is gazing for a longer time at a point on the briefing,the capture module 210 then taking a image of eyes of the speaker toobtain a visual focusing point of the speaker. In order to avoid anyerror caused by a movement of the speaker, and also to know more exactlythe eye direction of the speaker, the capture module 210 also obtainsthree eye focusing points (focus points) of the speaker. The three focuspoints of the speaker show as three points in the display device 30, thecapture module 210 then confirms a circle according to the three focuspoints of the speaker, and a center of the circle also can be confirmed,in one embodiment, the center of the confirmed circle is regarded as theeyes direction of the speaker.

FIG. 3 is a schematic illustration of a confirmed circle in differentsituations. Three examples can be seen, in example 1, the three focuspoints are from p1 to p2, then p2 to p3, obviously an area of a circlecomposed by the p1, p2, and p3 is small, which indicates that the eyesdirection of the speaker changes little. In example 2 and example 3, thethree visual focal positions are from p4 to p5, then p5 to p6, and fromp7 to p8, then p8 to p9, offsets of the points are relatively large,which indicates that the eyes direction of the speaker change much more.

Referring to FIG. 2, after the circle is confirmed according to thethree visual focal positions of the speaker, the detection module 220detects whether the area of the confirmed circle is less than a firstthreshold area. The first threshold area is a preset constant accordingto the speaker's preference, and stored in the storage system 250. Whenthe area of the circle is less than the first threshold, thetransmitting module 230 outputs the centre of the circle as the eyedirection coordinate. When the area of the circle is not less than thefirst threshold, the capture module 210 obtains further focus points ofthe speaker to confirm a new circle until the area of the new circle isless than the first threshold.

For example, referring to FIG. 3, assuming that the area of the circlein example 1 is less than the first threshold, indicating that thedirection of the speaker's eyes is substantially constant or close tothe coordinate, in other words, the eye direction location of thespeaker is the important content of the briefing, and the transmittingmodule 230 shows the eye direction location of the speaker in thedisplay device 30 so that the listeners realize the importance of thiscontent of the briefing. Assuming that the area of the circle in example2 is not less than the first threshold, which indicates that thespeaker's eyes move frequently, in other words, the present content ofthe briefing is not so important, the capture module 210 then obtainsfurther three visual focal positions of the user to confirm the newcircle. The example 3 is the same as example 2.

Referring again to FIG. 2, after the eye direction coordinate of thespeaker is shown in the display device 30, the direction(s) of the gazeof the speaker are made known to the audience in the following manner.

The transmitting module 230 of the video terminal 10 of the presentspeaker outputs the eye direction coordinate of that speaker to thevideo terminal 10 of the listeners. The displaying module 240 of thevideo terminal 10 of the listeners stores the eye direction coordinateof the speaker in the storage system 250. The following modules are allpart of or integrated with the video terminal 10 of each of thelisteners.

In at least one embodiment, the detection module 220 detects whether thestorage system 250 has stored a first previous eye direction coordinateof the presenter. If the storage system 250 has not stored the firstprevious eye direction coordinate of the speaker, then the eye directioncoordinate of the speaker is a newest eye direction coordinate and thedisplaying module 240 shows the eye direction coordinate of the speakerin the display device 30. If the storage system 250 has stored the firstprevious eye direction coordinate of the speaker, the detection module220 detects whether a distance between the eye direction coordinateoutput by the transmitting module 230 of video terminal 10 of thespeaker and the first previous eye direction coordinate stored in thestorage system 250 is less than a second threshold. The second thresholdis preset constant according to the speaker's preference, and stored inthe storage system 250. The displaying module 240 updates the firstprevious eye direction coordinate to the eye direction coordinate outputby the transmitting module 230 of video terminal 10 of the speaker whenthe distance between the eye direction coordinate transmitting module230 of video terminal 10 of the speaker and the first previous eyedirection coordinate stored in the storage system 250 is not less thanthe second threshold.

The foregoing disclosure of the various embodiments explains how thelisteners are made aware of the eye direction coordinate of the speaker,the following explains how the present speaker/presenter is made awareof the eyes directions of each of the listeners.

In at least one embodiment, when the eye direction of the speaker is apoint in the display device 30, the capture module 210 obtains the threevisual focal positions of the listeners to confirm a circle, then themethod of the modules of the video terminal device 10 detecting eyedirection of the listeners is the same as the method of the eyedirection of the speaker, and thus is not stated again. However, afterthe video terminal device 10 of the speaker receives the eye directioncoordinate of the listeners, the following method is applied.

In at least one embodiment, after the displaying module 240 of the videoterminal device 10 of the speaker receives the eye direction coordinateof the listeners, the detection module 220 detects whether the distancebetween the eye direction coordinate output by the transmitting module230 and a second previous eye direction coordinate is less than a thirdthreshold. The third threshold is a preset constant according to thespeaker's preference, and stored in the storage system 250. Thedisplaying module 240 shows the eye direction coordinate output by thetransmitting module 230 in the display device 30 to indicate an area atwhich the listeners are gazing when the distance between the eyedirection coordinate output by the transmitting module 230 and thesecond previous eye direction coordinate is less than the thirdthreshold. The displaying module shows the eye direction coordinateoutput by the transmitting module 230 in the display device to indicatethe area at which listener are gazing when the distance between the eyedirection coordinate output by the transmitting module 230 and thesecond previous eye direction coordinate is less than the thirdthreshold.

FIG. 4 is a flowchart of one embodiment of a method 400 of the videoterminal device transmitting eye direction target coordinates. Themethod 400 is applied for the video terminal device 10, and achieved bythe modules shown in FIG. 2.

In block S402, the capture module 210 continuously taking an images ofeyes of a user when the user gazes at a display device to obtain threevisual focal positions of the user, and confirming a circle according tothe three visual focal positions of the user.

In block S404, the detection module 220 detects whether an area of thecircle is less than a first threshold. If the area of the circle is lessthan the first threshold, the method 400 then proceeds to block S404, ifthe area of the circle is not less than the first threshold, the method400 then proceeds to block S402.

In block S406, the transmitting module 230 outputs a centre of thecircle as a eye direction coordinate. In another embodiment, thetransmitting module 230 also can output the eye direction coordinateshowing in the display device 30 or output the eye direction coordinateto other video terminal device.

When the area of the circle is not less than the first threshold, inblock S402, the capture module 210 continuously obtains again threevisual focal positions of the user to confirm a new circle until thearea of the new circle is less than the first threshold.

FIG. 5 is a flowchart of one embodiment of a method 500 of the videoterminal device 10 used by listeners handling the eye directioncoordinates of the speaker. The method 500 is applied for the videoterminal device, and achieved by the modules shown in FIG. 2. Indifferent embodiment, the listeners and the speaker can separately usethe video terminal device 10.

In block S502, the displaying module 240 stores the eye directioncoordinate of the speaker and shows the eye direction coordinate of thespeaker in the display device.

In block S504, the detection module 220 detects whether the storagesystem 250 has stored first previous eye direction coordinate. If not,then proceeds to block S508, otherwise proceeds to block S506.

In block S506, the detection module 220 detects whether a distancebetween the eye direction coordinate output by the transmitting module230 and the first previous eye direction coordinate stored in thestorage system is less than a second threshold. If not, the displayingmodule 210 updates the first previous eye direction coordinate to theeye direction coordinate, the method 500 then proceeds to block S508,otherwise the method 500 proceeds to block S510.

In block S508, the displaying module 240 shows the eye directioncoordinate output by the transmitting module 230 in the display device30.

In block S510, the displaying module 240 shows the second previous eyedirection coordinate in the display device 30.

FIG. 6 is a flowchart of one embodiment of a method 600 of the videoterminal device used by the speaker handling the eye directioncoordinates of the listeners. The method 600 is applied for the videoterminal device, and achieved by the modules shown in FIG. 2. Indifferent embodiment, the listeners and the speaker can separately usethe video terminal device 10.

In block S602, the displaying module 240 stores the eye directioncoordinate of the listeners and shows the eye direction coordinate ofthe listeners in the display device.

In block S604, the detection module 220 detects whether the distancebetween the eye direction coordinate output by the transmitting module230 and the eye direction coordinate of another user is less than athird threshold. Another user is the speaker. The If not, then themethod 600 proceeds to block S606, otherwise the method 600 proceeds toblock S608.

In block S606, the displaying module 240 shows a area which thelisteners are not earnest listening.

In block S608, the displaying module 240 shows a area which listener areearnest listening.

The foregoing disclosure of the various embodiments has been presentedfor purposes of illustration and description. It is not intended to beexhaustive or to limit the present disclosure to the precise formsdisclosed. Many variations and modifications of the embodimentsdescribed herein will be apparent to one of ordinary skill in the art inthe light of the above disclosure. The scope of the present disclosureis to be defined only by the claims appended hereto and theirequivalents.

What is claimed is:
 1. A video terminal device comprising a processorand a storage system, the storage system storing one or more softwareprograms in the form of computerized codes operable to be executed bythe processor, the one or more software programs comprising: a capturemodule, configured for continuously taking images of eyes of a user whenthe user gazes at a display device to obtain three visual focalpositions of the user, and confirming a circle according to the threevisual focal position of the user; a detection module, configured fordetecting whether an area of the circle is less than a first threshold;and a transmitting module, outputting a coordinate of a centre of thecircle as an eye direction coordinate when the area of the circle isless than the first threshold; wherein the first threshold area is apreset constant according to a speaker's preference.
 2. The videoterminal device of claim 1, wherein the capture module is furtherconfigured to continuously obtain again three new visual focal positionsof the user to confirm a new circle when the area of the circle is notless than the first threshold.
 3. The video terminal device of claim 2,further comprising: a displaying module, configured for storing the eyedirection coordinate output by the transmitting module in the storagesystem, and showing the eye direction coordinate in the display device.4. The video terminal device of claim 3, wherein the detection module isfurther configured to detect whether the storage system has ever storeda previous eye direction coordinate, and the displaying module shows theeye direction coordinate output by the transmitting module as a newesteye direction coordinate in the display device when the storage systemhas not stored the previous eye direction coordinate.
 5. The videoterminal device of claim 4, wherein the detection module is furtherconfigured to detect whether a distance between the eye directioncoordinate output by the transmitting module and the previous eyedirection coordinate stored in the storage system is less than a secondthreshold when the storage system has stored the previous eye directioncoordinate.
 6. The video terminal device of claim 5, wherein thedisplaying module configured to update the previous eye directioncoordinate to the eye direction coordinate output by the transmittingmodule when the distance between the eye direction coordinate output bythe transmitting module and the previous eye direction coordinate storedin the storage system is not less than the second threshold.
 7. Thevideo terminal device of claim 5, wherein the displaying module isconfigured to not update the previous eye direction coordinate when thedistance between the eye direction coordinate eye direction output bythe transmitting module and the previous eye direction coordinate storedin the storage system is less than the second threshold.
 8. The videoterminal device of claim 3, wherein the detection module is furtherconfigured to detect whether a distance between the eye directioncoordinate output by the transmitting module and a eye directioncoordinate of another user is less than a third threshold.
 9. The videoterminal device of claim 8, wherein the displaying module is furtherconfigured to show an area which listener are earnest listening when thedistance between the eye direction coordinate output by the transmittingmodule and the eye direction coordinate of another user is less than thethird threshold.
 10. The video terminal device of claim 8, wherein thedisplaying module is further configured to show an area which listenerare not earnest listening when the distance between the eye directioncoordinate output by the transmitting module and the eye directioncoordinate of another user is not less than the third threshold.
 11. Aneye direction detecting method of a video terminal device, the methodcomprising: continuously taking images of eyes of a user when the usergazes at a display device to obtain three visual focal positions of theuser, and confirming a circle according to the three visual focalpositions of the user; detecting whether area of the circle is less thana first threshold; and outputting a coordinate of a centre of the circleas an eye direction coordinate when the area of the circle is less thanthe first threshold; wherein the first threshold area is a presetconstant according to a speaker's preference.
 12. The eye directiondetecting method of a video terminal device of claim 11, furthercomprising: continuously obtaining again three new visual focalpositions of the user to confirm a new circle when the area of thecircle is not less than the first threshold.
 13. The eye directiondetecting method of a video terminal device of claim 12, furthercomprising: detecting whether a previous eye direction coordinate hasbeen stored ever; showing the eye direction coordinate as a newest eyedirection coordinate in the display device when the previous eyedirection coordinate has not been stored; detecting whether a distancebetween the eye direction coordinate and the previous eye directioncoordinate is less than a second threshold when the previous eyedirection coordinate has been stored; updating the previous eyedirection coordinate to the eye direction coordinate and showing the eyedirection coordinate as the newest eye direction coordinate in thedisplay device when the distance between the eye direction coordinateand the previous eye direction coordinate is not less than the secondthreshold; and showing the previous eye direction coordinate in thedisplay device when the distance between the eye direction coordinateand the previous eye direction coordinate is less than the secondthreshold.
 14. The eye direction detecting method of a video terminaldevice of claim 12, further comprising detecting whether the distancebetween the eye direction coordinate and a eye direction coordinate ofanother user is less than a third threshold; showing an area which atleast one listener is earnestly listening when the distance between theeye direction coordinate and the eye direction coordinate of anotheruser is less than the third threshold; and showing an area in which atleast one listener is not earnestly listening when the distance betweenthe eye direction coordinate and the eye direction coordinate of anotheruser is not less than the third threshold.
 15. A video terminal devicecomprising a processor and a storage system, the storage system storingone or more software programs in the form of computerized codes operableto be executed by the processor, the one or more software programscomprising: a capture module, configured for continuously taking imagesof eyes of a user when the user gazes at a display device to obtainthree visual focal positions of the user, and confirming a circleaccording to the three visual focal position of the user; a detectionmodule, configured for detecting whether an area of the circle is lessthan a first threshold; and a transmitting module, outputting acoordinate of a centre of the circle as an eye direction coordinate whenthe area of the circle is less than the first threshold; wherein thefirst threshold area is a preset constant according to a speaker'spreference, and the detection module is further configured to detectwhether a distance between the eye direction coordinate output by thetransmitting module and a eye direction coordinate of another user isless than a second threshold.
 16. The video terminal device of claim 15,further comprising: a displaying module, configured for storing the eyedirection coordinate output by the transmitting module in the storagesystem, and showing the eye direction coordinate in the display device.17. The video terminal device of claim 16, wherein the displaying moduleis further configured to show an area which listener are earnestlistening when the distance between the eye direction coordinate outputby the transmitting module and the eye direction coordinate of anotheruser is less than the second threshold.
 18. The video terminal device ofclaim 16, wherein the displaying module is further configured to show anarea which listener are not earnest listening when the distance betweenthe eye direction coordinate output by the transmitting module and theeye direction coordinate of another user is not less than the secondthreshold.